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ABSTRACT 

The National Assessment of Educational Progress uses 
a variety of test items and scoring techniques in measuring the 
writing achievement of three age groups — nine, thirteen, and 
seventeen year olds. This document discusses the holistic scoring of 
essays, including mechanical correctness and grammatical usage; the 
primary-trait method of scoring, with types of discourse rated 
according to their purposes; and the checking of content on a . 
prese^nce/absence basis. Included are tables of findings, examples of 
essays, background questionnaires, and scoring guides. (JM) 
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Highlights and .Trends from National 
A cessment : Writing and Change in Writing Skills 

by 

Ina V. S. Mullis 
National Assessment of Educational Progress 

Introduction 

There are many aspects of writing achievement and a variety 
of methods are used to score writing. What is important is that 
the items assessed and the scoring procedures used are appropriate 
to the measurement goals. For National Assessment, this meant 
using a variety of items and scoring techniques. Several essays 
were rated using holistic judgments — that is, in terms of their 
overall quality. These same essays were also scored for mechanical 
correctness and grammatical usage. The majority of essays and 
letters were scored using the primary trait method of scoring, in 
which types of discourse are rated according to their aim or purpose . 
For example, a letter written to a school principal about a change 
that should be made in the school should contain persuasive elements 
in order to accomplish its purpose. If it does not persuade, it * 
is not a good letter. Still other tasks were scored by checking 
certain content on a presence/absence basis. If the exercise asked 
for a business letter, knowledge about letter format was checked 
by noting the use or non-use of such conventions as greeting, closing 
or return address; 
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Writing Mechanics 

The first national assessment of writing was administered to 
17--year-olds in the spring of 1969 , to 13-year-olds in the fall of 
1969 and to 9-year-olds in the winter of 1970. The second assessment 
was administered four years later to 9- and 13-year-olds and five 
years later (spring of .1974) to 17-year-olds. Although both 
assessments consisted of a number of survey questions, multiple- 
choice and essays, one essay task included in both the 1969-70 
assessment and the 1973-74 assessment of writing was scored for 
mechanical correctness and grammatical usage of each age. Nine- 
year-olds wrote a story about the picture of a "Kangaroo" (Exercise 
102013) , while 13-year-olds and 17-year-olds were asked to describe 
"Something Worth Talking About" (Exercise ■ 203012) . The actual 
scoring guide used accompanies each of these exercises. 

Each paragraph was coded as either coherent or developed. At . 
the sentence level, scorers coded agreement errors and punctuation 
errors of both commission and omission. In addition, they cate- 
gorized each sentence by type (simple sentence , complex sentence 
with phrase, fragment, etc.). Awkward sentences received an 
additional code. Words were marked for misspelling, capitalization 
errors and word choice errors. The scoring and categorization at 
the paragraph and sentence le e s was done not only to be able to 
describe the types of errors respondents , made , but to be able to 
describe something about the level of complexity and sophistication 
of "the writing. 

The essays from both assessment years were scored by a group 
of English teachers, all with considerable experience in grammar 
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and linguistics • Each paper was scored by two readers and then 
reconciled by a third. After the paragraphs, sentences and words 
were coded, the essays and the codes were keypunched and the results 
were tabulated by computer. 

Holistic Scoring 

Descriptive scoring, like that discussed above, provides 
considerable information about essays, but it does not provide for 
a comparative evaluation of the quality of the essays. To determine 
the relative quality of the papers, they were read "holistically" — 
a term derived from this method's emphasis on a reader's response 
to the whole essay rather than to such aspects of it as style, 
content, mechanics and so on. Holistic scoring has long been used 
to evaluate essays written for the College Boards and, many other 
college entrance examinations. Experienced readers are trained by 
rating training papers exemplifying the range of score points until 
they have internalized the scoring system. They then read the papers, 
giving each a score from 1 to 8 based on their overall reaction 
to a paper and its relationship to the range of quality of the 
other papers. 

This method of scoring provides a reliable ranking of essays — a 
ranking most trained readers would endorse — but it tells us little 
about the papers, except that some are better than others. One 
never knows precisely why a paper received the rating it did. The 
scorers decide on the relative quality of each paper in comparison 
•to the other papers without any particular reflection on their 
decisions , • 
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Holistic scoring -.is difficult to replicate over time. Although 
the relative "ranking between the various papers may stay fairly 
constant, the aggregate rating for the total set of papers may 
change. Some papers are not clear examples of score points. In , 
these cases readers must decide, for example, whether a paper is 
a 4 or a 5. Depending on the readers and/or the time of the reading, 
systematic differences can occur in these decisions. In a NAEP study, 
we found that the same set of papers rescored holistically several 
years later had a significantly higher mean holistic score. 
Accordingly, NAEP decided not to conduct two holistic scorings at 
two different times. 

To reliably measure change in the quality of essays, NAEP 
mixed a sample of 1969 papers with a sample of 1974 papers and 
conducted a single scoring session. There was no way a scorer 
could detemmine whether he was reading a 1969 or a 1974 paper. 
Readers — all of them experienced English teachers — were trained 
and monitored in the usual fashion, working from folders containing 
papers from both assessment years. Each paper was ranked according 
to its overall quality. 

In the analysis, the mean holistic score of the 1969 papers 
was compared to that of the 1974 papers and the percentages of 
papers receiving each score point were also compared. For example, 
the percentage of 1969 papers rated 7 and 8 was compared to the 
percentage of 1974 papers rated 7 and 8. 

It can be 'argued that without mastery in the area of mechanics, 
a writer will not communicate successfully. Yet tallies of error 
rates only describe the level of technical competence exhibited 
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in the writing. It was felt that the evaluative approach of the 
holistic scoring system was complementary to the more descriptive 
method of scoring for writing mechanics. Both systems were used 
in order to characterize specific writing accomplishments more fully 
than either approach could have done alone • 

Detailed results from the writing mechanics scoring and the 
holistic scoring are published in Writing Mechanics, 1969-1974/ 
Report 05-W-Ol . The overall results and some samples of 13-year-old 
and 17-year-old essays not published in the report are presented 
below. 

Overall Results 

1. The quality of the average essay written by a 9-year-old 
remained much the 'same between 1970 and 1974. 

2. At age 13, there was an overall decline in the quality 
of the essays written for the second assessment. There 
was a movement toward shorter, simpler expression. Also, 
there was a marked increase, particularly among males, 

in rambling prose, i.e., somewhat unfocussed writing 
containing more run-on sentences and more awkwardness 
than was evident in 1969. 

3. At age 17, there was an overall decline in the quality 

of the essays written for the second assessment. Increases 
in awkwardness, run-on sentences and incoherent paragraphs 
most likely reduced the overall quality of the essays. 
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. Samples 

Although some of the essays written for the 1973-74 assessment 
were excellent and £ few were totally incoherent, the remainder 
suggest that the average writer is uneasy with the conventions 
of written language. So even though the essays exhibited a great 
range of ability, the conclusion' was reached that at age 13 and 
age 17 there was slippage in overall coherency. The following 
are samples of the types of writing that helped contribute to the 
decline in coherency. Both 13-year-olds and 17-year-olds were 
asked to describe something worth talking about. 

17-Year-Olds 

- THE BERKELY PIT HAS GOT TO BE THE BIGGEST HOLE IN THE 
GROUND EVER IT KEEPS THE TOWN GOING WITH ALL THE COPPER AND 

■ °^HE^It'''eMPLOYEYS°MANY MONTANA WORKERS. IT'S A GOOD PROFESSION 
TO GO INTO IF YOU LIKE EXCITEMENT THE PIT' MUST HAVE THE BIGGEST 
TRUCKS AND EQUIPMENT IN THE WORLD. 

THERE IS REALLY ONLY ONE THING WRONG WITH THE PIT ,AND THAT 
IS IT IS TEARING MORE AND MORE OF ARE TOWN DOWN, TO KEEP 
ADVANCING AND GETTING MORE COPPER AND OTHER MINERAL DEPOISTS . 

I ONLY HOPE THAT IN THE FUTCURE ALL OF BUTTE DOESN'T GET 
TORE DOWN FOR THE PIT OR THE ANACONDA COMPANY. . 

THIS YEAR WILL BE THE lOOTH RUN FOR THE ROSES, AND VERY 
MANY FAMOUS PEOPLE WILL ATTEND BECAUSE OF THAT. 

•THE DERBY IS FOLLOWED BY ANOTHER RACE LATER ON CALLED THE 
PREAKNESS, AND THEN BELMONT STAKES. 

FEW HORSES HAVE WON ALL, THREE THAT ARE CALLED THE TRIPLE 
CROWN, BUT RECENTLY SECRETARIAT WON IT LAST YEAR AND IN MY 
OPINION, IS THE GREASTEST HORSE EVER. 

I'VE TRAVELED THE APPALATIAN TRAIL IN PENNSYLVANIA AREA AND 
ENJOYED THE EXPERIANCE SO MUCH THAT I AM PLANNING TO RETURN 

^°I^ENJ0YED THE FACT THE THE TRAILS WERE FREE OF LITTER AND 
DISPITE THE POPULARITY OF THE AREA THE ANIMAL POPULATION 
SEEMED HIGH. 

WHEN we' WERE DOWN IN SANDEGRO WE SEEN THE STADIUM THERE AND 
MY UCULE HODES UP THE PHONES AND THING FOR FOOTBALL GAMES AND 
WE GO TO GO INSIDE TO SEE IT. ITS A PRETTY BIG PLACE I THINK 
HE SIAD IT HOLDS 50,000 PEOPLE OR WHAT EVERY AND THERES ABOUT 
4 TO 5 STORYS HIGH IT MADE OUT OF CEMENT AND THERE A GREAT 
BIG PACKING LOT OUT FRONT AND THE FEILD IS THAT FAKE GRASS. 
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IT MOVES LIKE A SNAKE AS IT SWIFTLY FLOWS AROUND THE BENDS . 
THOUGH MANY A ROCK AND SUCH ABSTICALS IT KEEPS ITS FLOWING 
MOTION. DEEP WITHIN ITS STOMACH IT HIDES THE SECRET OF LIFE. 
FILLED WITH THE GREENESS OF PLANT LIFE, WHERE ONE CAN FIND 
FISH HIDDEN IN THEIR SHADOW. 

MY FATHER HAS SOME LITTLE BABY CATTLES NOW. SOME DIE AND 
SOME DON'T, BUT HE HAVE HAD PRETTY GOOD LUCK SO FAR. LAST 
WINTER THERE WAS A TERRIBLE SNOW STORM, AND MY FATHER LOST A 
LOT OF BABY CALVES. THE LAND IS GREEN, AND AS YOU COME IN 
THERE IS A GATE YOU HAVE TO UNLOCKED IT, BEFORE YOU CAN COME 
IN. ON THE GATE THERE A NO TRESSPASSIN SIGN ON THE GATE. AS 
YOU COME IN THERE IS A LITTLE PATH YOU CAN DRIVE THOUGH TO 
, GO TO PONDS. AND BEFORE YOU GO, HERE IS A BIG RED BARN AND 
YOU GO BY THAT THIN THERE IS A SOME PONDS. THERE IS 3 OF THEM. 

AND HE ALSO HAS A SHELTERBELT TOO. -IN THE SUMMER TIME IT 
IS BEAUTIFUL. IT IS JUST LIKE A PICTURE. YOU NEVER CAUGHT 
NOTHING YET, WHILE I HAV'NT. BUT MY FATHER SAID, "THERE IS 
FISH IN THE FISH POND". BY THE WAY, I LIVE ONLY 15 MILES 
• AWAY FROM THE FARM. 

THIS WHAT I 'AM ABOUT TO DESCRIBE IS IN ATLANTA IT IS USE FOR 
SPORT IT HAS A BIG HOLD IN THE TOP ON THE GROUND IT HAS GRASS 
IT IS VERY LARGE. IT IS USE YEAR ROUND, IT ALSO HAVE A LAGE 
FLIELD IT IS THE SPORT PLACE IN ATLANTA FOOT BALL, AND BASEBALL ' 
OR PLAY THERE SOMETIME OF SPORT OR PLAY THERE, SOMETIME , 
THOUSAND OF PEOPLE GO THERE TO HEAR BILLY GRAHAM IT IS IN 
THE MIDDLE OF SOME HIGHWAYS. IT HAVE A RE^ LARGE PARKING LOT. 

THE THING I CHOOSE TO WRITE ABOUT IS THE MT RUSHMORE 
MONUMENT IS SOUTH DAKOTA. MT RUSHMORE IS A MONUMENT OF 
SOLID STONE SKETCHED IN MASSIVE PORTIONS IN THE ROCK OF THE 
BLACK HILLS IN SOUTH DAKOTA. THE M0NUEI4NT HONORS FOUR GREAT 
MEN, ALL PREDIDENTS OF THE UNITED STATES. THE PRESIDENTS 
HONORED ARE GEORGE WASHINGTON, ABRAHAM LINCOLN, THOMAS 
JEFFERSON, AND THEODORE ROOSEVELT. THE MONUMENT WAS BEGUN 
IN THE EARLY 1900 'S BY GUSTAVE BORGLUIN, A SWEDISH IMMIGRANT. 
BECAUSE HE HAD JUST OBTAINED HIS UNITED STATES CITIZANSHIP, 
HE WANTED TO DO SOMETHING GREAT TO HONOR HIS NEW COUNTRY. ^ 
WHILE VACATIONING IN SOUTH DAKOTA, THE IDEA CAME TO HIM TO 
BUILD A MASSIVE MONUEMNT IN STONE TO THE FOUR AMERICANS HE 
-RESPECTED AND HONORED MOST. HE CHOOSE 3 PAST PRESIDENTS; 
GEORGE WASHINGTON, THOMAS JEFFERSON AND ABRAHAM LINCOLN. 
HE ALSO CHOSE' THE PRESEDENT OF THE U.S. AT THE TIME THE " • 
MONUEMNT WAS BEGUN, THEODORE. ROOSEVELT, TO INCLUDE IN HIS 
MEMORIAL MONUMENT. THE MONUMENT WAS BEGUN IN THE 1920 'S. 
GUSTAVE WORKED NIGHT AND DAY TO SEE HIS MONUMENT FINISHED. 
MAYBE JUST A LITTLE TOO HARD, FOR IN THE 1940 'S WITH HIS CREATION 
STILL UNFINISHED HE DIED. HIS SON, THEREFORE HAD TO FINISH 
THE MASSIVE WORK. THE MONUMENT WAS COMPLETED A FEW YEARS LATER 
AND DEDICATED BY FRANKLIN ROOSEVELT, A COUSIN OF THEODORE. 
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THEJ MONUMENT SITS IN THE HEART OF THE BLACK HILLS SURROUNDED 
BY TALL, MASSIVE, EVERGREEN TREES. THE MONUMENT IT SELF IS 
CHI SLED RIGHT OUT OF THE MOUNTAIN AND IS A MASSIVE SIGHT TO 
BEHOLD. IF -EVER YOU FELT SMALL, YOU SURE DO WHEN YOU ARE BY ' 
THAT MONUMENT. ALSO, EVERY YEAR THEY BEGIN THE MASSIVE 
PROJECT OF CLEANING THE FOUR MEN. A THRILL TO SEE 1 



13-Year~01ds 

AT THE BOTTOM OF THE^^GRAND CANYON YOU CAN CAMP OUT AND EXPLORE 
IT, IT DOESN'T HAVE WATER IN IT NOW BECAUSE IT WAS DRIED UP 
IN THAT SPOT BUT THE LITTLE COLOR-:ODO RIVER IS STILL RUNNING. 

A DOG IS A ANEMAL AND DOES HAVE A BRAN. HAVES BIG TEETH, 
A NOSE THAT HE CAN SMELL WITH. " 

A DOG, IT COME IN ALL SIZE, A DOG WAKE ON FOUR LEGS. A 
DOG HAVE TWO EYE, HE HAS EARS AND HAS HAIR. THIS DOG CAN 
SEE GOOD AT NIGHT. 

DECRIBE THE ARCH IN ST. LOUIS' NEAR THE RIVER IN ST. LOUIS 
NEAR THE ROBERT E. LEE IT TALL AND IT CURVES WHEN YOU GET AT 
THE TOPNP IT SCARES YOU AND WHEN YOU ENTER THE ELLVATOR'S 
IT HAS. ROOM FOR FIVE AND IT GOES 20 MILES PER HOUR. , AND THE 
ROBERT E. LEE IS A RESTURANT YOU HAVE TO HAVE A RESIVATION 
ON TOP DECK IT SERVES ON THE BOTTEM THEY SERVE STEAKS AND 
THINGS LIKE THAT AND THEY HAVE A BOX ON THE LEFTSIDE OF THE • 
BOAT. 

EVEN THOUGH THE NIAGARA FALL'S IS VERY BIG AND BEAUTYFULL 
IT IS GRADULY-DISTROYING IT'S, SELF . 

BY EVERY TON OF WATER EVERY DAY IT IS GRADUALY WEARING 
AWAY. THE NIAGARA FALL'S IS A VERY BIG AND BEAUTIFUL FALL'S 
BUT THEY ARE NOT THE ONLY FALL'S THERE IS THE - HORSESHOE 
FALLS, AND THE STRAIGHT FALL'S. THE FALL'S ARE NOT ONLY 
TO LOOK AT BUT YOU CAN GO WRITE INTO THE FALL'S AND BE IN 
SIDE OF IT. IVE NEVER BEEN IN THE NIAGARA. FALL'S BUT I HAVE 
SEEN THEM BEFORE. WHEN YOU GO IN THE FALL'S YOU MUST HAVE A 
RAIN COAT . . 

YOU CAN RENT ONE IF YOU HAVE ONE. BUT NIAGARA IS NOT THE 
ONLY ONE TO LOOK AT BUT THEY HAVE A RESTRANT THAT AT LEAST 
10 STORIES, AND YOU CAN GO ON A ELEVATOR TO THE TOP. 

THE DOGWOOD BLOSSOMS ARE SO SOFT AND PRETTY THE DOGWOOD 
BLOSSOM HAVE FOUR-LEATLIY AND IT'S SHARP LIKE A HEART & HAS 
AT THE END OF THE HERAT SHARP THERE IS A BROWN SPOTS DOWN IN 
THE MIDDLE OF THE HEART. 

THE TREE IS SO SMALL AT IT LOOK LIKE IT COULD HOLD ALL THOSE . 
FLOWERS. THE BRANCHE .' RE SMALL AND ALMOST WHITE WITH 
BROWN LINES A THE TREb & BRANCHES. 

WHY I PICK THE DOGWOOD TREE IS ITS BEAUTY & THE STORY 
BEHIND IT. 
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IT WAS A CALM/ COGL DAY WHEN I SAT AMONG THE ROUGH GROOVES 
OF THE SILVER-LINED ROCKS LOOKING AT THE REFLECTION OF THE 
RED, ORANGE AND YELLOW LEAVES ON THE TREES OF THE GOLDEN 
FOREST. THE COOL, PLACID POND WAS SURROUNDED BY ROCKS AND 
TREES. THERE NO LONGER ARE THE GREEN, SUPPLE LEAVES ON THE 
TREES, BUT FALLING ARE THE DRY, CRACKED LEAVES. I FEEL THE ' 
COOL BREEZE BLOW ACROSS MY FOREHEAD. THERE IS ONLY THE COLORLESS, 
DEAD GRASS AND DRIED UP WEEDS LEFT OF WHAT WAS A GREEN PARADISE 
IN THE SPRING AND SUMMER MONTHS. A FLOCK OF BIRDS IN. A 
V SHAPE FLEW OVERHEAD. THE SUN IS HIDING BEHIND THE CLOUDS. 
IN THIS AREA NOT A CREATURE IS STIRRING BUT YET EVERY NOW 
AND THEN THE CHIRPING OF A CRICKET WAS HEARD. THIS PLACE 
SEEMED AS THOUGH IT WERE AN ENCHANTED FOREST BECAUSE OF ITS 
MAGICAL APPEARANCE. THIS DIVINE PARADISE IS HEAVEN TO ME. 

TO ME THE GRAND CANYON IS A HUGE PIT OR CANYON DUG BY THE 
COLOADO RIVERNP IT TOOK MANY' THOUSANDS OF YEARS THE RIVER -TO 
DIG OUT THIS. CANYON. 

'THE GRAND CANYON IS LAYERS OF ROCK. IT MAY DIP UP AND DOWN. 
FOR INSTANCE IT MAY BE LOWER ON THE EDGE OF THE CANYON THAN 
IT IS IN THE MIDDLE. 

TO ME THIS CANYON IS A HUGE PIT, MADE BY A RIVER, DUG IN 
MANY LAYERS OF ROCK. 

IT IS A TALL GREEN STACYON OF A WOMAN HOLDING A TORCH IN 
RIGHT HAND. YOU CAN GO IN IT. PEOPLE CAN GO OUT IN A FAIRY 
BOAT AND GO AROUND . OR EVEN GO UP IN IT . ' 



Background Ques-bionnaire 

The following questions were administered to all in-school 
17-year-olds in the 1973-74 assessment. Although, the background 
variable analysis of this data has not yet been completed, the 
raw results are quite interesting. 
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1973-74 NATIONAL ASSESSMENT 
BACKGROUND QUESTIONNAIRE ADM 
IN-SCHOOL 17-YEAR-O; 



How many reports and essays have you 
last six weeks as part of any school 



# of Reports Percent 

or Essays Response 

0 13% 

1 11% 



2 

3 
4 
5 



16% 
■15% 
11% 
9% 



6-10 17% 
more than 10 8% 



1) About one'-fourth of the respondei 
no writing in 6 weeks. 

2) Three-fourths of the respondents 
one writing assignment per week. 



12 



In the general English, literature or grammar classes 
you have taken during the past two years, about what 
part of the class time was spent on instruction in how 
to write reports and essays? 



None of the time 

Little of the time 

About one-third of the time 

About one-half of the time 

Most of the time 



Almost half the respondents reported that. little or no 
time in their English classes has been spent on instruction 
in how to write. 



5% 
42% 
33% 
14% 

6% 



In addition to the general English, literature or grammar 
classes you have taken during the last two years, have 
you had or are you now taking any of the following courses 
concerned with how to write? 



Cre^ative writing course 20% 

Remedial writing course 6% 

Composition 8% 

Journalism 3% 

Other writing course 4% 
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Do you write a paper more than once before you turn 
in to your teachers? 



53% Usually 

40% Sometimes 

5% Never 

1% I haven't written any papers. 



When your papers are returned, do they have written 
suggestions on how to improve your writing? 

33% Usually 

56% Sometimes 

10% Never 

1% I haven't written any papers. 



/ 
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Primary .Trait Scoring 

Although the holistic method works very well to separate the 
better papers from the poorer papers, National Assessment has had 
some trouble in the past explaining what the results actually 
mean. For one thing, very little discussion about the overall 
quality of the papers is possible. Is it that the poor papers 
represent competent writing and the better papers are excellent or 
is it that the poor papers are practically illiterate and the better 
papers are just "less poor"? In other words, no one but the actual 
readers really know if the respondents as a whole could write at 
an acceptable level. Also, score points are almost impossible to 
describe. Obviously a "7" is better than a "5" yet National 
Assessment could not report in concrete terms what constitutes the 
difference between those papers without further analysis. 

In order to overcome this problem, to assure replicability of 
assessments and to provide more specific information to educators, 
NAEP has developed a "primary trait" scoring system. 

- The rationale underlying primary trait scoring is that writing 
is done in terms of an audience and can be judged in view of its 
effects upon that audience. Particular writing tasks require 
particular approaches and no others if they are to be successful. 
The approach used by the writer to reach and affect his audience . 
will be the most important — the primary — trait of a piece of writing. 
The writer of a set of directions must present things in a logical 
and unambiguous manner if he expects readers to follow the directions. 
The primary trait of a written set of directions would be an 
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unambiguous/ sequential and logical progression of instructions. 
Successful papers will have that trait, unsuccessful papers will 
notr regardless of how clever or well written they may be in other 
respects. The purpose of campaign literature is to persuade a • ' .. 
reader to vote for a candidate/ A successful campaign paper will 
have certain persuasive traits that an unsuccessful one does not 
possess, 

A carefully constructed testing situation provides opportunities 
for respondents to demonstrate their ability to choose and effectively 
carry out appropriate rhetorical strategies, but the effects of 
the strategies cannot be measured by actual success. For example, 
it is not feasible to have respondents write real job applications 
and then check to see if ifhey were hired. An evaluation of the 
writing has to be based on its likelihood of ^achieving the desired 
effect. The features that will contribute to this success must 
be identified and defined in terms of their importance. These 
definitions then become the scoring criteria. The primary trait 
score point essentially indicates whether or not a sample of 
writing contains the traits it must have in order to accomplish 
its purpose. 

Although the number of score points can vary with the exercise. 
National Assessment has found that a four-point scale is preferable. 
Usually a "1" indicates absence of the primary trait, "2" indicates 
presence of the primary trait, "3" indicates competence* and "4" 
indicates excellence. The following are some guidelines for scoring 
primary traits for the three basic types of writing. 

EKLC 



In broad terms, the primary trait for persuasive writi'^ig is 
presentation of logical and compelling arguments. Generally, then 
a "1" paper would present no reasonable arguments, a "2" would 
have one reason or perhaps two not particularly well thought out 
reasons, a "3" would be a logically thought out presentation 
containing several reasons and a "4" paper would support the logical 
reasons with compelling details. Of course, mpre precise definitions 
of these score points would be needed in terms of the actual task. 

Explanatory writing should present facts in a clear and 
orderly fashion. Therefore, score points can move from a response 
with no clarity and logical order and one end of the scale to a 
totally unified presentation at the other end of the scale. Of 
course, each exercise needs a guide for rating responses to the 
particular situation created.. For example, a response to an 
explanatory exercise usually must include some type of specific 
information in order to receive one of the higher score points. 

As mentioned earlier, expressive tasks can have a variety of 
aims and as such have a variety of primary traits. Each guide has 
to be written in terms of that primary trait. Two primary traits 
NAEP scored for were imaginative expression of feeling through a 
point of view and elabbration of role. The key to defining score 
points for expressive writing usually is found in the quantity 
and quality of elaboration. The lower score points generally have 
no or little elaboration. The higher score points are more imagina- 
tive papers, since the premise presented in the response is supported 
by vivid and inventive details. 
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It should also be remembered in developing primary trait 
scoring guides that the amount of information that can be obtained 
from a single writing task is only limited by imagination, zeal 
and resources. Almost any writing task can be categorized in a 
variety of ways. For example, information about use of tense, 
point of view, tone, used, revisions and mechanics can be gathered 
in addition to the. primary trait. 

The scoring of the primary traits was done by experienced 
English teachers. Sample papers were used to train readers to 
evaluate the writing in terms of the scoring guides . For most 
exercises/ each paper was rated by two readers and in cases of 
disagreement reconciled by a third. 

National data from one creative writing exercise scored using 
the primary trait method are below. A copy of the exercise, an 
abbreviated version of the scoring guide, and several sample 
responses follow. 
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CHILDREN ON A BOAT 



Primary Trait: Entry into the Imaginary 

World of the Picture 





1 


2 


3 4 




AGE 9 


22% 


58% 


10% <1% 




AGE 13 


10% 


57% 


29% 2% 




AGE 17 


10% 


35% 


44% 10% 








Secondary Traits 












• 






Past 


Hypothetical Cannot 


be Determined 


AGE 9 


38% 


29% 


8% 


16% 


AGE 13 


26% 


46% 


2% 


24% 


AGE 17 


25% 


53% 


<1% 


20% 


POINT OF 


VIEW 










Child 


Observer 


Cannot be Determined - 




AGE 9 


22% 


65% 


4% 




AGE 13 


29% 


67% 


2% 




AGE 17 


35% 


62% 


1% 





ERIC 



DIALOGUE 

AGE 13 
AGE 17 

AGE 9 

AGE- 13-"' 

AGE 17 



One Person 
6% 
8% 
6% 

USED FANTASY 
5% 
16% 
23% 



Two Persons 
6% 
7% 
3% 

INCLUDED INSIGHTS 
<1% 
1% 
5% 



Look cai'efully at the picture. 



These kids are having fun jumping on the overturned boat. Imagine you are 
one of the children in the picture. Or if you wish, imagine that you are some- 
one standing nearby watching the children. Tell what is going on as he or she 
would tell it. Write as if you were telling this to a good friend, in a way that 
expresses strong feehngs. Help your friend FEEL the experience too. Space 
is provided on the next three pages. 
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CHILDREN ON THE BOAT SCORING GUIDE 



PRIMARY TRAIT - ENTRY INTO THE IMAGINARY WORLD OF THE PICTURE 



1 = 



2 = 



NO ENTHY INTO THE IMAGINAHY WORLD OF TH2 PICTURE 
Respondents write about children, the boat, or ibout anything 
else to do VI th the picture. However there is only a single 
statement or the information is too disjointed to make 'a 
point. Random details, bits of information, or lists of 
observations that do not create a situation. Also include 
pure description, papers that only report what's in the 
photograph or picture. 

ENTRY INTO THE IMAGINARY WORLD OF THE PICTURE. 
Respondents have accepted the world of the picture. However 
the control and consistency necessary to create a structured 
presentation are lacking. Often there is no structure to the 
world of the picture. This is evidenced by few internal 
transitions and details that don't harmoriize. The ideas may 
be related, but don't make a whole. Lack of consistency and 
transitions result in little logical progression of ideas. On 
the other hand, these papers may have structure, but th*^- 
narrative is not worked out. There is little imagination on 
the part of the writer to create the story. An attitude may 
be stated, but it's not illustrated. You do not "feel" a mood 
• has been created. Other papers may have soma structure and 
the outline of story or the hint of a mood, yet neither are 
developed. Generally "2" papers are either undeveloped or 
developed in a helter-skelter or confusing manner. 

3 = GOOD. ENTRY INTO THE IMAGINARY HORLD OF THE PICTURE. 

Papers are generally competent. .Respondents evidence control 
and consistency to create a structured presentation. often 
there are strong topic sentences and good transitions. 
However, the good guality of the papers is marred by 
development that is skeletal, or somewhat uneven. Narratives 
- display imagination, but often are left with gaps or other 
uneyeness. Details may be inadeguate, excessive, or unclear. 
Attitudes may be stated and somewhat supported, but 
necessarily presented to "help your friend feel." 
A definite mood is not created. The underdeveloped paper has 
a definite beginning and end yet there is not enough material 
to fill out the structure or it is . contrived- The unevenly 
developed papers either have excessive details or the 
structure is oddly filled out. 
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tt = EMOTIVE AND CONSISTENT ENTRY INTO THE IMAGINAHY WORLD OF THE 
PICTUBE. 

These papers are structurally whole. Loose ends have been 
tied up or cut off (although a strong paper without closure 
can be rated in this category) . Papers are consistent. 
Narratives are well and evenly developed or attitudes are 
expressed so a definite mood is created. You ^ do "feel" the 
experience. The structure is unified and supported by 
imaginative and evocative • details. 

SECONDARY TRAITS ^ 
FANTASY (scored present or not present) 

Story tries to reproduce the fantasy games of children, such as 
pretending to be pirates, ship wreclced, or riding whales. Stating the 
fantasy is not enough, two or more unelaborated fantasy situations, or 
a fantasy situation with at least one elaboration must be present. 
Descriptions of games, for example, follow the leader or king of the 
hill are not included. Fantasy is more than a literal and logical 
explemation of the picture. 

INSIGHTS (scored present or not present) 

Story develops insights into the writer's perspective on his life or 
life in general. The message should be integral to the story - not 
tacked on sentiment. The generalizations or social commentaries can 
be stated implicitly as well as explicitly. Themes are often based on 
positive or negative value statements about childhood or adulthood. 
Reminiscence (I remember when...) is not enough, some meaningful or 
intelligent statement must be implied. 
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J = DIRECT QUOTE FROM ONE PERSON IN THE STORY. THE ONE PERSON MAY 
TALK MORS THAN ONCE. WHEN IN DOUBT WHETHER TWO STATEMENTS ARE 
MADS BY THE SAME PERSON OH DIFFERENT PEOPLE, CODE 1. A DIRECT 
QUOTE OF A THOUGHT ALSO COUNTS. CAN BE IN HYPOTHETICAL TENSE. 

2 = DIRECT QUOTE FROM TWO OR MORE PERSONS IN THE STORY. 

3 = DOES NOT USE DIALOGUE IN THE STORY. 
POINT OF VIEW 

1 = POINT OP VIEW IS . CONSISTENTLY ONE OF THE FIVE CHILDREN. 

INCLUDE "IF I WERE ONE OF THE CHILDREN..." AND RECALLING 
PARTICIPATION AS ONE OF THE CHILDREN. 

2 = POINT OF VIEW IS CONSISTENTLY ONE OF AN OBSERVER. WHEN AN 

OBSERVER JOINS THE CHILDREN IN PLAY, THE POINT OF VIEW IS 
STILL "2" BECAUSE THE OBSERVER MAKES A SIXTH PERSON PLAYING. 
INCLUDE PAPERS WITH MINIMAL EVIDENCE EVEN WHEN DIFFICULT TO 
TELL WHICH POINT OF VIEW IS BEING TAKEN. 

3 - POINT OF VIEW CANNOT BE DETERMINED, OR DOES NOT CONTROL POINT 

OF VIEW. 

TEIm'SE •■ ' ■ 



-1 = PRESENT TENSE - PAST TENSE E^-Y ALSO BE PRESENT IF NOT PART OF 
,'■ THE "MAIN LINE" OF THE STORY. 

2 = PAST TENSE - IF A PAST TENSE DESCRIPTION IS ACCEPTABLY BROUGHT 

UP TO PRESENT, CODS AS "PAST." SOMETIMES THE PRESENT IS USED 
TO CREATE A FRAME FOR PAST EVENTS. CODE THIS AS " PAST, SINCE 
THE ACTUAL DESCRIPTION IS' IN THE PAST. 

3 = HYPOTHETICAL TIME - PAPERS WRITTEN ENTIRELY IN THE "IF I WERE 

ON THE BOAT" OR "IF I WERE THERE, I WOULD." THESE PAPERS 
OFTEN INCLUDE FUTURE REFERENCES SUCH AS "WHEN I GET ON T'HE 
BOAT I WILL." IF PART IS HYPOTHETICAL AND REST PAST OR 
PRESENT AND TENSE IS CONTROLLED, CODE PRESENT OH PAST. IF THE 
INTRODUCTION, UP TO TWO SENTENCES, IS ONLY PART IN PAST OR 
, PRESENT THEN CODE HYPOTHETICAL. 

U = CANNOT DETERMINE TIME, OR DOES NOT CONTROL TENSE. (ONE WRONG 
. TENSE PLACES THE PAPER IN THIS CATEGORY.) EICEPT DROWNED T:'. 
THE PRESENT. 
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SAMPLE RESPONSES 



PRIMARY TRAIT 

1 = NO ENTRY INTO THE IMAGINARY WORLD OF THE PICTURE 



SECONDARY TRAITS 



POINT 
OP 

PANTASr INSIGHTS DIALOGOS VIEW TENSE 

2 2 3 2 1 

9 The children are on top of the boat walking around. They are 

trying to balance themselfs so they won't fall, one of them, 
is balancing sitting down on the boat. 



2 2 3 2 1 

13 The children are sliding on the boat they are siting and 

sliding^ and they are standing and sliding on it. The 
children are having fun sliding on it. One of them are 
standing. one of them are sliding backword. one of them are 
running on it. they see other boats around them. One of them 
are stolping on it* The children are little and they think 
that it is fun. They are by the lake. The boat is on the 
dock were you tie the boat on to. they have warm clothing on 
them. 



2 2 .3 2 1 

17 the kid are haveing fun playing on the boat. 

trying to see who can walk on the boat without falling. 

The child siteing down is rocking the boat to make it even harder* 
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2 = ENTRY INTO THE li-lAGINARY WORLD OF THE PICTURE 



SECONDARY TRAITS 



POINT 
OP 

FANTASY INSIGHTS DIALOGOE VIEW TENSE 



2 2 2 .2 3 

I would qo over to them and asJc them if I can join them p.nd I 
would ask them yhat they are playing I would play thcc game 
wif- •""^r" "e would h3 7e fun playing with each other. I would 
say ca . >l you might fall and hurt yourself They would say 

we are playing a game and we are having alot of fun playing 
the game wi each other. 



1 2 3 3 4 

Well if I were one of the children I would probably be having 
fun. The children are jumping up and down on the boat. They 
.are also sliding down along the side of the boat. The 
children are also sitting on the boat. tfe pretended like we 
were sailing in a boat. One of the kids was sitting on the 
front of the boat. The other little boy was standing up like 
a sail. 



2 - 2 3 2 4 

This happen yesterday, their we five kid on a boat, they were 
all having alot of fun- and if you were there you probaly you 
play with them. Like they were juirping up and down and make 
believe they were sailing it . upside down. They were like 
jumping off and on. It was very great to be so small and do 
the same thing there did. 
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GOOD ENTRY INTO THE IMAGINARY 



FANTASY 



IKSIG 



2 



2 



I am' going to tell about i 
. playing, one day me and my 
of as and ve vanted- to 
wont Down to the lake and 
and they wanted to go p 
them not to But they didn" 
But then Sandy Shelly S 
and said hey Lori come on . 
just for a Little while;, an < 
ffhile and then X said X ar 
my friends hey you guys ar 

and they said olc and 
something and the boat 
happend and X said to them 
been in the water come on ' 
home and they nerer went c 
neathec did X 

The end 



As X was standing or 
saw five children jumping i: 
have been having fun .for tl 

X sat and watched then 
the sadden one of the ktds 
water • 

At first X started to 
already climbed out of 
returned to the house and i 

The first' time X remen 
how yoa are reading thi 
that kid that fell into the 



Yesterday when we wer 
Steve Lori# Sue# Jody and 
a chilly day so we didn* t 
While .going for a walk, w 
the dock an overturned boa 
boat whilinq we were strug 
This was difficult beca 
from side to side. Xt was 
of the mountian gone, 
boy. 

But he got in lots of 
Bven if. he did win, winnin 
otsr fan in such away. 



2 



2 



2 



2 
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■ > POIHT 

OP 

PANTAST IKSIGHTS OlALOGDE VIEW TENSE 



17. 



- 3 



It was clear that Jimmy wanted to be captain, -but i 
init about to let him, ^It's mr boat: why should I let him 
--captaxn? For the fi-st time in a lonq while i sat back in 
the hot, dry sand and tocit a loolc at the dock, the lagoon, -.and 
^ 1^°^^^^® I call Ji^nie in the sunmer It's a "whole different 



wasn 

be 




, , ' • potion that made 

people more relaxed jnore agreeable i glanced up at the hot 
^un, then down on the the way it made the water glisten and 
played with theJwaveiS, so that they seemed to give off * light 
also/ I looked bacK Jit; Jiinmy sitting impatiently at the heliJl 
Of the overturned rovboat although the paint was chinoing off 
the wood Ijas rotted, noQne could' imagine the memories the 
Kana ii»» held for me. So many great fishing exoeditions, 
picnics on sand bars; U seemed all the happiness I had as a 
child centered arooni this beautiful cottage this broken 
rowboat, I turned, facing the lagoon, and looked at what had 
once been my whole lifQ ^ , . 



. Jumping and running on the boat is very enioyable, Up we 
jump and dpwn we float. X feel as if I could sail the boat 
around the world and back. The salty sea air blows through my 
nostrils. ay .body faels engulfed in this exotic salt 
concoction The vind beats against my cheeks 

The white, glistening, enamal underside of the boat feels 
like silk to the touch. The trees are alivo, pulsatino 
watching our childish games. 

I feel like I could pl^y forever. No concept of time, no 
' responsibilities, no stresses encourage my exuberance. 

ay body has separated from my yoirit. i am no longer 
encaged in a prison of bones and skin. These are no barriers 
now. I can do whatever I w^nt, whenever I want to do it! . 

12 2 1 1 " 

^'^ »hee, isn't this fun. Lets imaging we're outcast 

5 f!u^^; .f"^^ ^^^^ ^^i^' tie the brigging. This is our 

faithful ship, sailing? over the tropical seas. Peel the 
strong, salty wind whisking over your face! r can almost feel 
the waves shoving under our feet. How all lump up! There we 
just missed getting dunked by that big breaker. How's the 
weather up their at the top of the crows mast. Jack. Just 
tine? Good. Kow let5 be off on our way on. another adventure, 
tets go tc the arctic this time. Su, its getting mighty cold 
now. Look? rheras a giant whale!. Lets spear him and take it 
back to cur homeland. X can even, hear its heaving lungs. 
Closer, closer, nov. Good^ we ^ot her. Pull her in. What, 
she.'s pulling us. Hold on! Pull harder! »ow, we're gaining 
speed. Wheres she going to take us? tfhats going to become of 
these poor lost sculs in this perilous situation. ^ Well stay 
tuned to tnis program, next week for further adventures. Same 
time, same channel See you now: this is tne hearty Kaptin Kidd 
signing off now. And remember, if you bad breath and rotton 
teeth »ise goopv. Goopy brand ^ tooth paste is the most 
fantastic product now on the market, if your eyes look kind 
of soupy,, use goopy. 

Mow that was fun, what shall we do now. sail for 
fantastic Australia weil Its off now! 
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